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ABSTRACT 

The purpose of this study was to explore the value of 
variables beyond undergraduate grades and admission test scores in 
predicting success in the preclinical curriculum of the College of 
Human Medicine of Michigan State University (East Lansing) . Subjects 
included about 93 students with relatively low scores on the Medical 
College Admissions Test (MCAT) . Admissions folders were located for 
students who entered the University between 1979 and 1984 with MCAT 
scores less than 8. The number of quarter hours " incomplete " and 
types/numbers of courses from which students "withdrew" were 
determined via an analysis of undergraduate transcripts. The written 
comments of admissions interviewers were reviewed, and notations 
regarding four characteristics (motivation, analytic skill, 
self -organization skill, and skill in maintaining a clear focus on 
study or on other valued activities) that received more than 
occasional comment were noted. Science MCAT scores and total 
undergraduate grade point average, "no credit" grades received in the 
preclinical curriculum, scores from initial attempts on Part I of the 
examination of the National Board of Medical Examiners (NBME) , and 
the delay (in months) between June of the second year of 
matriculation and date when NBME Part i was first attempted were 
analyzed. R v suits indicate that non-academic variables derived from 
admissions interviews provide very limited gains in prediction. One 
figure and four data tables are included. (TJH) 
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PURPOSE 



The purpose of this study was to explore the value of variables beyond 
^ undergraduate grades and admissions test scores in predicting success in the preclinical 
SO curriculum of the College of Human Medicine of Michigan State University for a group 
0} of students with relatively low scores on the Medical College Admissions Test (MCAT). 

O 

^ BACKGROUND 
CO 

Q During the admissions process professional schools may evaluate some personal 

and social characteristics which are not strongly associated with academic success, in an 
attempt to assure the presence of desirable professional characteristics in their graduates. 
For example, in the admissions process for the College of Human Medicine of Michigan 
State University an assessment has been made of 1 tolerance of ambiguity," "self appraisal 
skills," and "life problem solving" among other characteristics of interest in prospective 
students. Indicators (such as undergraduate grades and aptitude test scores) of the 
ability to learn the content of the profession's subjects are also important in the 
admissions decision, but not infrequently a candidate will present both very desirable 
non-academic characteristics and academic indicators that raise questions about the 
prospects of academic failure during training. There is a substantial published literature 
(e.g., Nowacek et al, 1987), indicating that admissions test scores and undergraduate 
grades both provide some prediction of medical students' academic performance, 
particularly their performance in the tirst two year: ;f medical school. Together, MCAT 
scores and indices based on undergraduate grades can usually provide a multiple R of .5 
to .7. However, once a well-classified set of undergraduate grades and the full range of 
admissions test scores have been used for predi don, little more is typically gained from 
further use of test scores or prior grades. 
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If prediction of academic success could be improved, fewer admitted students 
would later be acknowledged, after much "wear-and-tear" on the student and the 
institution, to have failed and fewer candidates for admission would be mistakenly 
rejected because of concerns over their prospects for academic success. Two alternative 
sources of predictors of academic adequacy were of interest, given the data available on 
students admitted to MSLPs College of Human Medicine. A number of investigators 
have suggested use of a range of nonacademic variables projected to determine the 
deployment of scholastic abilities (e.g., Tracey and Sedlacek, 1985). Jackson and 
Dawson-Saunders (1987) found that the likelihood of academic difficulty in the first year 
at Southern Illinois University School of Medicine was associated with the frequency of 
withdrawal from or receipt of an "Incomplete" grade in undergraduate courses. 

The records of admissions interviews for MSLPs College of Human Medicine 
often contain written reactions and comments by the interviewer. Many of the written 
comments are related to a range of characteristics and experiences of the candidates that 
are not rated but that might be related to academic performance. The undergraduate 
transcript contains the information needed to assess the frequency of withdrawals and 
incompletes in undergraduate courses. Thus, the admissions folder of admitted College 
of Human Medicine students, when linked to a set of indices of medical school 
performance, can provide the data needed to assess the probable predictive value of a 
candidate set of non-academic variables, and of frequency of non-completion of 
undergraduate courses. Because some of the most troublesome admissions decisions 
involve otherwise attractive students with low values of standard academic predictors, the 
study was focused on students whose average MCAT scores were less than 8. MCAT 
scores are reported (Mitchell, 1987) to be most predictive within this group of students. 



METHODS 

Admissions folders were located for students who entered MSU's College of 
Human Medicine between 1979 and 1984 and whose MCAT average score was less than 
8. In each folder the number of term (quarter) hours "incomplete" and "withdrew" in 
undergraduate transcripts were determined. In addition, the written comments of 
admissions interviewers were reviewed and a record made of notations (both positive and 
negative) regarding four characteristics that received more than occasional comment. 
The four were: motivation to be a student of medicine (indexed by references to 
persistence, initiative, and special effort in pursuing studies relevant to medicine), 
analytic skill or disposition (indexed by references to quality of thought, analysis, or 
problem solving in the interview), skill or disposition in self-organization (indexed by 
references to practical prioritization or specific achievement gained through 
organization), and skill or disposition in maintaining a clear focus on study or on ether 
valued activity (indexed by references to goal-directedness and resistance to diffusion of 
effort). Examples of notations recorded in each category are displayed in Figure 1. The 
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notations in each category were counted, and the counts normalized to correspond to two 
admissions interviewers (the typical number). For each student in the group, Science 
MCAT scores and total undergraduate grade point average were lso entered. If 
available, the score from the student's first tiy on Part I of the examination of the 
National Board of Medical Examiners, and the best score, if different, were also 
recorded. A note was made of the delay (in months) between June of the second year of 
matriculation and the date when NBME Part I was first attempted. Finally, a record was 
made of the number of "N" (No credit) grades received in the preclinical curriculum. 

To reduce the number of predictors, variables that were logically related or that 
were both conceptually and empirically related (Pearson r's of .3 or greater) were 
combined. This produced the following set of predictors whose value was being tested: 

•The number of course credit hours unfinished within the term of enrollment as an 
undergraduate (#_UNFIN), the sum of the hours incomplete and withdrew 

•The number of positive notations less the number of negative notations related to 
motivation (MOHV) 

-The number of positive notations less the number of negative notations related to 
analytic skill/disposition (ANALYTIC) 

•The number of positive notations less the number of negative notations related to 
personal organization (ORGANIZ) 

-The number of positive notations less the number of negative notations related to 
personal focus (FOCUS) 

The frequency of agreement between interviewers in the number of notations in a 
specific area was noted, and compared to the agreement that would be expected if each 
interviewer had produced notations randomly at a rate consistent with the observed 
number of notations. 

For each of four variables indexing an important aspect of academic performance 
in the preclinical curriculum (first score on NBME Part I--NBME1ST; best score on 
NBME Part I-NBME TOP; the delay between June of the second year and the date of 
first try on Part I-DELAY; the number of N grades~#_OF_N'S) a regression analysis 
was performed. MCAT Science scores (MCAT_SQ) and total undergraduate GPA 
(TOTGPA) were always entered first, so the question asked was always whether 
inclusion of the other variables added significantly to the prediction available from these 
commonly used predictors. 

Data on NBME scores, MCATs, undergraduate GPA and the number of 
undergraduate course credits unfinished were available for 93 students. However, for 8 
of these students the number of N grades had not been recorded, and for a different 15 
the admissions rating data could not be located. Thus the sample size for different 
regression analyses changed with the outcome index and the predictor variables that 
were being examined. 
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RESULTS 



The correlations among the indices of performance are shown in Table I. Aside 
from the correlation between the two sets of NBME scores, the strength of association 
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among the performance indices is moderately small, indicating their relative 
independence. The number of N's received is positively related to the length of the 
preclinical program (Delay) and negatively related to the NBME scores, as expected. 
The veiy large correlation between the two sets of NBME scores can also be expected, 
since the scores are different only for students who took Part I more than once. 

Correlations between the outcome indices and the predictors, and those among 
predictors, are shown in Table II. For each performance index there is a single best 
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predictor, and it is a summary of earlier behavior of the same kind. The number of 
unfinished undergraduate credits correlates strongly with the length of students' 
preclinical programs, and negatively, and more weakly, with the NBME scores. 
Correlations with variables derived from the notes of admissions interviews were weak 
(never greater than .3), although they were generally in the expected direction. 
Agreement in frequency of notation between interviewers was not significantly different 
from what would be expected by chance for any of the variables derived from notations 
made by admissions interviewers. The notations counted in these variables were fairly 
rare events, so the mean of each of the variables was less than 1. 

The contribution to prediction made by the number of unfinished undergraduate 
credits is summarized in Table HI, where the impact of adding #_UNFIN to the 
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prediction equation is shown for each of the performance indices. Knowledge of the 
number of undergraduate credits withdrawn from or caken as "incomplete" significantly 
improves the prediction of students' delay in taking NBME, Part I, and of students' best 
Part I scores. The F associated with adding #_UNFTN to the prediction of students' first 
scores on Part I is greater than 1, but not large enough for statistical significance (with 
alpha = .05). 

Table IV records those instances in which there was an F greater than 1 
associated with adding to prediction a variable based on notes made by admissions 
interviewers. Three (MOTTV, ORGANIZ, and FOCUS) of the four variables developed 
from admissions interview notations were related to a delay in taking NBME, Part I, and 
the FOCUS variable makes a significant contribution to prediction of that delay even 
after Science MCAT, undergraduate GPA, and the number of unfinished undergraduate 
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course hours (see below) have been entered in the prediction equation. Notations 
related to motivation produced an F greater than 1 in predicting the number of N grades 
received, but did not improve that prediction significantly (alpha = .05). 



DISCUSSION 

Ihe patterr of correlations observed here is consistent with the general rule that 
"like predicts like." Correlations among test scores are highest, undergraduate GPA is 
the best single predictor of the number of N grades received in the preclinical 
curriculum, and undergraduate withdrawals and incompletes (actions that delay 
completion of the undergraduate curriculum) correlate best with delays in completing the 
preclinical curriculum. 

The results of this study support and extend those of Jackson and Dawson- 
Saunders(1987); tendency to leave courses uncompleted as an undergraduate is a useful 
predictor of behavior as a medical student and adds significantly to the ability to predict 
the best NBME Part I score of students who enter medical school with low MCATs. 
The set of performance indices examined in this study is not the same as the set studied 
by Jackson and Dawson-Saunders, but in each case tendency to leave courses 
uncompleted is an indicator of an unwanted outcome. Faced with an attractive 
candidate with an equivocal prediction of success from the undergraduate GPA and 
MCAT scores, a medical school admissions committee may well consider an acceptance 
if the undergraduate record shows few incompletes and withdrawals. Conversely many 
instances of failure to complete undergraduate courses may suggest a rejection if the rest 
of the record is not decisive. 

In contrast to the apparent utility of measures jf unfinished undergraduate stuay, 
the nonacademic variables derived from the admissions interviews have produced very 
limited gains in prediction. These four variables, measured as they were, add only 
marginally in predicting performance indices for the preclinical curriculum. However, 
the weakness of these variables as predictors may stem from inconsistencies of 
measurement due to the post hoc definition of the variables and to their assessment over 
a set of observations and records that were not focussed on and hzd no systematic 
connection to the constructs of the variables. (Admissions interviewers were not asked 
to specifically attend to or record observations related to the areas central for each of 
the variables studied here; interviewers were trained to focus on and rate a set of 
behavioral tendencies expected to anticipate applicants* style of medical practice.) The 
variables of this study would almost certainly be better assessed if each interviewer were 
instructed to ask questions during the interview, about issues inherent in the variables 
and if interviewers had been brought to a common framework indicating what incidents 
or reactions were worth recording and which were not. While the variables, as 
measured, don't add significantly to prediction, their pattern of correlation with the other 
variables makes sense. Thus, it seems likely that if the variables were better assessed 
they would prove to be more useful adjuncts to prediction. 
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Motivation (persistence, initia tive, special effort 

Biochem research on weekends 
Little energy or enthusiasm (-) 

Analytical (quality of thought analysis, problem solving) 

Able to pick out important facts and use them 
Often went on a tangent (-) 

Organization ( practical prioritizing, achievement thru 
organization) 

Able to prioritize 

Ability for self-instruction weak (-) 

Focus (Goal-directedness. resistance to diffusion) 
Gear about goals 

Figure 1. Examples of notations made by interviewers 
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TABLE I 

Correlations Among Performance Indices 

# of Ns Delay NBMEJst 
Delay .25 

NBMEJst -.36 -.31 

NBME Top -.23 -.44 .81 
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TABLE II 

Correlations Among Predictors and With Performance Indices 



MCAT_Sci Tot_GPA #_Unfin MOTIV ANALY ORGANIZ FOCUS 

#ofNs -.21 -.44 .02 -.20 -.02 .04 .03 

Delay -.28 -.23 .51 -.17 .07 -.24 -.22 

NBMEJst .52 .37 -.27 -.02 .17 .05 .03 

NBME_Top .56 .24 -.33 .09 .10 -.06 .13 

MCAT_Sci .22 -.15 .02 .01 -.04 .00 

TotGPA -.27 .14 .17 .14 .03 

#_Unfin -.03 -.12 -.20 -.05 

MOTIV .26 -.03 .17 

ANALY .14 .06 

ORGANIZ .09 

(Sample size = 70) 
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TABLE III 



Predictions of Performance Indices In Which 
Number of Unfinished Credits Is Added to the Predictors 



Prediction of # of Ns (Sample size = 85) 
Entry 



Order Variables 


E 


E 


12 


1 a Z MUA1 oCl & lot OrA 

3 #_Unfin 


.42 
.43 


O A 

8.9 
<1 


<.001 


Prediction of Delay (Sample size = 93) 








Entry 

Order Variables 


£ 


F 


12 


1 A 2 MPAT Sri * Tnt HPA 

3 #_Unfin 


.47 


o.z 
11.3 


•UUJ 

<.005 


Prediction of NBME lst (Sample size = 


93) 






Entry 

Order Variables 


E 


F 


12 


1 & 2 MCAT Sci & Tot GPA 
3 #_Unfin 


.55 
.56 


19.1 
Z4 


<.001 
>.05 


Prediction of NBME Top (Sample size = 


: 93) 






Entry 

Order Variables 


E 


F 


12 


1 & 2 MCAT Sci & Tot GPA 
3 #_Unfin 


.48 
.53 


13.7 
6.2 


<.001 
>.02 
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TABLE IV 



Prediction of Performance Indices in Which 
Addition of a Variable Based on Interview Notes 
Is Associated with an "F Greater Than One 



Prediction of Delay (Sample size = 78) 
Entry 



Order Variable 


E 


£ 


12 


1 - 3 MCAT Sci & Tot GPA & # Unfin 


.47 


6.9 


<.001 


4 FOCUS 


.51 


4.5 


<.05 


Prediction of # of Ns (Sample size = 70) 








Entry 








Order Variable 


£ 


E 


C 


1 - 2 MCATSci & Tot_GPA 


.45 


8.6 


<.001 


3 Motiv 


.47 


1.7 


>.05 
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